November  1995 


Statistically  Optimum 
Pre  -  and  Post  Filtering  in  Quantization 

Jamal  Tuqan  and  P.  P.  Vaidyanathan 
Department  of  Electrical  Engineering 


Prefilter 


Postfilter 


x{n) 


G{e^“) 


y{n) 


QS 


z(n) 


x(n) 


A  general  pre  -  and  post  filtering  scheme 


19970415  026 


dtic 


quality 


California  Institute  of  Technology 
Pasadena,  CA  91125 

DISTRIBUTION  STATEMENTIl^ 

Approved  for  public  releasej 
Distribution  Ublimitedi 


STATISTICALLY  OPTIMUM  PRE-  AND  POST  FILTERING  IN  QUANTIZATION  t 


11/28/95 


Jamal  Tuqan,  Student  member,  IEEE  and  P.P.  Vaidyanathan,  Fellow,  IEEE 
Department  of  Electrical  Engineering  116-81 
California  Institute  of  Technology 
Pasadena,  CA  91125,  USA. 

Phone:  (818)  395-4681  Fax:  (818)  564-9307 
Contact  author:  P.  P.  Vaidyanathan,  E-mail :  ppvnath@sys.caltech.edu 


Abstract.  We  consider  the  optimization  of  pre-  and  post  filters  surrounding  a  quantization  system.  The 
goal  is  to  optimize  the  filters  such  that  the  mean  square  error  due  to  quantization  is  minimized  under  the  key 
constraint  that  the  quantization  noise  variance  is  directly  proportional  to  the  variance  of  the  quantization  system 
input.  Unlike  some  previous  work,  the  postfilter  is  not  restricted  to  be  the  inverse  of  the  prefilter.  With  no 
order  constraint  on  the  filters,  we  present  closed  form  solutions  for  the  optimum  pre-  and  post  filters  when  the 
quantization  system  is  a  uniform  quantizer.  Using  these  optimum  solutions,  we  obtain  a  coding  gain  expression 
for  the  system  under  study.  The  coding  gain  expression  clearly  indicates  that,  at  high  bit  rates,  there  is  no  loss 
in  generality  in  restricting  the  postfilter  to  be  the  inverse  of  the  prefilter.  We  then  repeat  the  same  analysis  with 
first  order  pre-  and  post  filters  in  the  form  1  -f-  and  1/(1  -I-  72“^)-  In  specific,  we  study  two  cases  :  (a)  FIR 
prefilter,  HR  postfilter  and  (b)  HR  prefilter,  FIR  postfilter.  For  each  case,  we  obtain  a  mean  square  expression, 
optimize  the  coefficients  a  and  7  and  provide  some  examples  where  we  compare  coding  gain  performance  with  the 
case  of  Q  =  7.  In  the  last  section,  we  assume  that  the  quantization  system  is  an  orthonormal  perfect  reconstruction 
filter  bank.  To  apply  the  optimum  pre-  and  post  filters  derived  earlier,  the  output  of  the  filter  bank  must  be  WSS 
which,  in  general,  is  not  true.  We  provide  two  theorems,  each  under  a  different  set  of  assumptions,  that  guarantee 
the  wide  sense  stationarity  of  the  filter  bank  output.  We  then  propose  a  suboptimum  procedure  to  increase  the 
coding  gain  of  the  orthonormal  filter  baink. 
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/.  INTRODUCTION 

Consider  the  general  scheme  shown  in  Fig.  1.1  where  the  box  labeled  QS  represents  a  quantization  system. 
The  input  sequence  x{n)  is  passed  through  a  prefilter  G{e^^)  and  produces  an  output  y{n).  The  sequence  y{n) 
is  then  quantized  and  filtered  with  a  postfilter  to  reproduce  an  estimate  of  the  input  denoted  by  x{n). 

The  quantization  system  QS  can  be  a  simple  uniform  queintizer  or  a  more  sophisticated  quantization  system  such 
as  the  ilf-channel  uniform  subband  coder  (SBC)  shown  in  Fig.  1.2.  Assuming  that  the  quantization  system  is 
constrained  to  have  a  budget  of  b  bits,  the  main  theme  in  this  paper  is  to  jointly  optimize  the  prefilter  and 

the  postfilter  H such  that  the  mean  square  value  E{e^  (n)}  of  the  reconstruction  error  where  e(n)  =  x{n)—x{n) 
is  minimized. 

The  renewed  interest  in  the  above  classic  problem  was  motivated  by  its  relation  to  some  issues  in  the  area 
of  subband  coding.  To  elaborate  more,  consider  the  M-channel  uniform  SBC  of  Fig.  1.2.  The  boxes  labeled 
Q  represent  subband  quantizers,  a  set  of  uniform  quantizers  which  are  modeled  by  additive  noise  sources.  An 
equivalent  representation  of  the  uniform  SBC  is  given  in  Fig.  1.3.  It  consists  of  two  matrices  and  R(e*^^), 

known  respectively  as  the  analysis  and  synthesis  polyphase  matrices.  In  the  absence  of  quantizers,  the  filter  bank 
(FB)  is  said  to  have  the  perfect  reconstruction  (PR)  property  if  and  only  if  R(e-^^)  =  E"^(e^‘^)  [1].  A  perfect 
reconstruction  filter. bank  (PRFB)  is  also  known  as  a  biorthogonal  FB.  An  important  subclass  of  uniform  PR 
filter  banks  is  the  class  of  orthonormal  or  paraunitary  (PU)  filter  banks.  In  this  case,  the  analysis  polyphase 
matrix  exhibits  the  lossless  property,  mathematically  expressed  as  E(e-^‘^)E'^(e*^‘^)  =  I  V  u;,  where  the  superscript 
t  denotes  the  conjugate  transpose  operation.  By  choosing  the  synthesis  polyphctse  matrix  R(e-^‘^)  to  be  equal  to 
E‘^(e'f‘*^),  perfect  reconstruction  is  guaranteed. 

In  the  presence  of  quantizers,  perfect  reconstruction  is  not  possible  because  quantization  is  a  lossy  process. 
The  FB  output  x(n)  in  this  case  is  the  original  input  x{n)  plus  a  filtered  version  of  the  quantization  noise  denoted 
by  v{n).  Recently,  several  authors  have  considered  optimization  of  filter  banks  when  quantizers  are  present  [2], 
[3],  [4],  [5],  The  goal  is  to  find  a  set  of  analysis  filters  such  that  the  mean  square  reconstruction  error  at  the  FB 
output  is  minimized.  For  the  sake  of  further  discussions,  we  will  from  now  on  refer  to  the  problem  of  optimizing  a 
FB  with  the  presence  of  quantizers  as  the  subband  coding  problem.  In  a  parallel  fashion,  interest  in  the  so  called 
energy  compaction  problem  was  growing  [6],  [7],  [8].  Although  the  energy  compaction  problem  might  at  first 
seem  decoupled  from  the  subband  coding  problem,  Vaidyanathan  [9]  recently  showed  that  the  energy  compaction 
problem  and  the  subband  coding  problem  for  the  ceise  of  an  orthonormal  SBC  are  actually  highly  connected.  In 
fact,  it  turns  out  that  the  solution  of  both  problems  is  the  same  for  an  M  channel  orthonormal  PRFB.  Such 
filter  banks  are  referred  to  as  optimum  orthonormal  filter  banka.  We  will  use  some  of  the  results  of  optimum 
orthonormal  filter  banks  later  in  section  V. 

Although  the  subband  coding  problem  was  carefully  analyzed  and  solved  for  the  class  of  orthonormal  FB 
[ideal  filter  case],  the  M  channel  [M  ^  1]  maximally  decimated  optimum  biorthogonal  FB  remains  for  example  an 
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open  problem.  Only  the  solution  of  the  one  channel  case  is  well  established  [10].  Furthermore,  it  is  well  known 
[11]  that,  in  the  presence  of  quantizers,  the  synthesis  polyphase  matrix  is  not  necessarily  the  inverse  of  the  analysis 
polyphase  matrix.  Restricting  ourselves  to  the  class  of  biorthogonal  FB  when  quantizers  are  present  is  therefore 
a  loss  of  generality.  A  similar  observation  was  given  by  Kovacevic  [2]  for  the  case  where  the  subband  quantizer  Q 
is  modeled  as  a  Lloyd-Max  quantizer.  While  the  synthesis  bank  was  optimized  in  [11],  Vaidyanathan  and  Chen 
did  not  address  the  issue  of  optimizing  the  analysis  bank.  Furthermore,  their  allocation  of  subband  bits  was  done 
before  optimizing  the  synthesis  bank. 

The  joint  optimization  of  the  analysis  bank  and  the  synthesis  bank  together  with  the  allocation  of  subband 
bits  is  quite  a  challenging  problem.  In  this  paper,  we  will  provide  a  joint  optimum  solution  of  the  pre-  and  post 
filters  for  the  special  case  of  M  =  1.  The  system  of  Fig.  1.1  when  the  quantization  system  QS  is  a  uniform 
quantizer  can  indeed  be  seen  as  the  one  channel  case  of  the  more  general  and  difficult  M  channel  problem.  It  is 
also  a  generalization  of  the  so-called  half-whitening  scheme  [10]  where  the  postfilter  is  assumed  to  be  the  inverse 
of  the  prefilter.  A  summary  of  all  the  paper’s  results  is  given  below. 

1.1.  Brief  Overview  of  Past  Related  Work 

The  problem  of  finding  optimum  pre  and  postfilters  around  a  noisy  processor  has  been  considered  by  various 
researchers  especially  in  the  field  of  communication  theory.  Costas  [12]  has  jointly  optimized  pre  and  post  filters 
over  an  analog  communication  channel  subject  to  a  power  constraint  on  the  prefilter.  Chan  and  Donaldson  [13] 
considered  the  same  problem  with  the  input  to  the  postfilter  sampled  every  T  seconds.  Berger  and  Tufts  [14] 
optimized  transmission  and  receiving  filters  in  PAM  communication  systems  to  minimize  the  mean  square  error 
distortion  resulting  from  channel  noise  and  intersymbol  interference.  Malvar  and  Staelin  [15]  offered  an  iterative 
algorithm  to  design  FIR  pre-  and  postfilters  in  the  presence  of  a  downsampler  and  an  upsampler. 

The  first  fundamental  difference  between  the  above  problems  and  the  quantization  problem  under  study  in 
this  paper  is  the  nature  of  the  noise  variance.  In  specific,  we  will  always  assume  throughout  this  paper  that  the 
noise  variance  is  directly  proportional  to  the  variance  of  the  input  to  the  quantization  system.  Such  a  constraint 
describes  in  a  fairly  accurate  manner  the  interaction  between  the  quantization  system  granular  noise  output  and 
the  dynamic  range  of  the  quantization  system  input  process.  A  simple  example  would  be  the  relation  tr,  =  c2 
used  in  [10]  for  the  case  of  a  uniform  quantizer.  In  a  communication  problem  setting,  the  noise  source  variance  is 
always  assumed  to  be  independent  of  the  channel  input  signal  statistics.  The  second  main  difference  is  that,  in  a 
communication  problem,  the  prefilter  is  usually  power  constrained.  This  is  not  the  case  for  the  quantizer  problem. 

Taking  a  different  approach  than  the  one  used  in  communications,  Jayant  and  Noll  analyzed  the  case  where 
the  quantization  system  QS  is  a  simple  uniform  quantizer  and  the  postfilter  is  simply  the  inverse  of 

the  prefilter,  i.e.,  =  1/G(e^’").  Applying  the  Cauchy-Schwartz  inequality,  the  magnitude  response  of  the 

optimum  filter  can  be  found  to  be  \Gopt{e^^)\  =  The  system  was  therefore  called  the  half  whitening 
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scheme  [10]  and  represents  an  optimum  one  channel  biorthogonal  FB.  Recently,  Djokovic  and  Vaidyanathan  [16] 
repeated  the  analysis  for  the  case  where  the  quantization  system  QS  is  a  uniform  orthonormal  FB. 

I, 2.  Main  Results  and  Outline  of  the  Paper 

1.  In  the  early  sections  of  this  paper,  we  will  assume  that  the  quantization  system  QS  is  a  uniform  quantizer. 
With  similar  assumptions  as  the  one  used  by  Jayant  and  Noll  in  the  derivation  of  the  half  whitening  solution, 
we  derive  optimum  solution  for  the  more  general  scheme  of  Fig.  1.1.  In  specific,  closed  form  expressions  for 
the  optimum  ideal  pre-  and  post  filters  are  derived  in  section  II. 

2.  In  section  III,  using  the  optimum  pre-  and  postfilters  of  section  II,  we  derive  an  expression  for  the  so  called 
coding  gain  of  the  scheme  of  Fig.  1.1.  The  beauty  of  this  expression  is  that  it  clearly  indicates  that  there  is 
no  loss  of  generality  in  using  the  half  whitening  scheme  if  we  are  quantizing  at  high  bit  rate,  a  result  that  is 
intuitively  very  appealing. 

3.  In  section  IV,  we  repeat  the  same  type  of  analysis  with  first  order  pre  and  post  filters  with  monic  polynomials. 
We  derive  an  expression  for  the  mean  square  error  for  the  cases  of  (a)  FIR  prefilter,  IIR  postfilter  and  (b)  IIR 
prefilter,  FIR  postfilter.  We  then  provide  some  examples  where  the  coefficients  of  the  filters  can  be  computed 
numerically.  We  compare  the  coding  gain  of  such  cases  with  the  one  obtained  from  a  first  order  one  channel 
biorthogonal  FB.  Our  results  indicate  again  that  unless  we  are  quantizing  at  a  very  low  bit  rate,  the  solution 
of  the  more  general  scheme  of  Fig.  1.1  tends  to  the  half  whitening  scheme. 

4.  In  section  V,  we  assume  that  the  quantization  system  QS  is  an  orthonormal  uniform  PRFB.  We  do  not  however 
try  to  generalize  the  scheme  proposed  by  Djokovic  and  Vaidyanathan.  Instead,  we  propose  a  suboptimum 
procedure.  We  first  develop  two  theorems  that  give  sufficient  conditions  for  wide  sense  stationarity  of  the 
output  noise  of  a  non-uniform  orthonormal  PRFB.  We  then  apply  the  optimum  pre-  and  post  filters  of  section 
II  at  the  input  and  output  of  the  FB  respectively  to  improve  the  performance  of  the  original  orthonormal 
PRFB. 

II.  OPTIMUM  UNCONSTRAINED  PRE-  AND  POST  FILTERS 

The  main  goal  of  this  section  is  to  jointly  optimize  the  prefilter  0(6^^^)  and  postfilter  of  Fig.  1.1  [QS 

is  a  uniform  quantizer]  to  minimize  the  mean  square  error  =  E{x{n)  —  x{n)}  subject  to  the  constraint 

=  02-2^2  (2.1) 

where  (7g  is  the  quantization  noise  variance,  c  is  a  constant  that  depends  on  the  statisticeJ  distribution  of  y{n)  and 
the  overflow  probability,  and  is  the  variance  of  the  quantizer  input.  Our  main  assumptions  for  this  section  are 
summarized  as  follows  : 

1.  All  random  processes  are  zero  mean,  real  and  jointly  wide  sense  stationary. 

2.  The  input  x{n)  and  the  quantization  noise  q{n)  are  uncorrelated  processes,  i.e.,  E{x{n)q{7n)}  =  0  V  n,m. 
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3.  The  quantization  noise  q{n)  is  white  with  variance  as  in  (2.1). 

4.  The  filters  and  G(eJ")  are  not  constrained  to  be  rational  functions,  i.e.,  the  optimum  and 

G{e-’“)  can  be  ideal  filters.  Furthermore,  no  causality  constraint  is  imposed. 

5.  The  power  spectral  density  is  positive  for  all  w.  Furthermore,  when  deriving  the  optimum  solution 

for  the  prefilter,  we  will  also  require  and  its  first  derivative  to  be  continuous  functions  of  frequency. 


2. 1.  The  Optimum  Post  Filter 

To  develop  optimum  closed  form  solutions  for  both  filters,  we  first  fix  the  prefilter  and  optimize 

The  optimum  post  filter  solution  is  given  in  the  following  theorem. 


Theorem  2.1.  For  a  fixed  preBlter  G(eJ"),  the  optimum  posthlter  Hopt{e^‘^)  is  the  well-known  Wiener  Biter  and 
is  given  by : 


Hcptien  - 


Sxxien 


^0-26  FTT  2(i<; 


(2.2) 


Proof.  For  a  fixed  prefilter  ©(e-i"),  the  input  to  the  postfilter  H(e^“)  is  a  filtered  version  of  the  desired  signal 
embedded  in  quantization  noise.  This  is  a  classical  Wiener  filtering  setting  and  hence,  the  optimum  postfilter 
is  given  by  [17]  Hopt{^‘^)  =  5*^(e^'‘^)/5„(e^“)  where  z(n)  =  y{n)  +  q{n)  is  the  noisy  input  to  the  wiener  filter. 
Since  x{n)  and  9(n)  are  assumed  uncorrelated,  it  is  easy  to  see  that  Sxz{€^‘^)  =  =  G{ei^)  Sxxiy^  )  and 

Szz{e^'^)  =  Syy{ei^)  +  c,  =  |G(e-’‘^)|^5ii(e-'")  +  where  the  *  denotes  complex  conjugation.  Substituting  in  the 
above,  we  get 

Giei'^ySxxje^^')  ^  1  .  Sxxje^^)  ^2.3) 

j2  G(eJ^)  (To 

«  ^  Sxx{ei‘^)  + - - — 2 


Hoptien  = 


|G(eJ‘^)f5xi(e-'")  +  (T|  G(eJ‘^) 


Substituting  the  constraint  (2.1)  in  this  last  equation,  we  obtain  the  above  solution.  ■ 

The  optimum  postfilter  can  be  drav-n  as  in  Fig.  2.1.  The  Wiener  filter  of  (2.2)  is  therefore  expressed  as  a 

cascade  of  two  filters  :  The  first  filter  is  the  inverse  of  the  prefilter  G(e-'").  Its  output  is  simply  the  original  input 

a;(n)  embedded  in  a  filtered  version  of  the  quantization  noise  process.  The  power  spectral  density  of  the  filtered 

2 

Quantization  noise  process  is - - — o  •  The  second  filter  is  the  optimum  IViener  filter  for  the  output  of  the  inverse 

\G{ei-)t 

filter. 

Using  the  optimum  post  filter  solution  (2.2)  and  the  constraint  (2.1),  we  can  now  derive  an  expression  for  the 
mean  square  error  only  in  terms  of  the  prefilter  G(e^‘*'). 


£  =  F{e^(n)}  =  E{e{n)  ■  (x(n)  -  x(n))} 

=  E{e{n)  •  x(n)}  =  E{{x{n)  -  i(n))  •  x(n)}  ^2.4) 

OO  00 

=  .Rw(0)-  Yi  Hk)  ■  E{xin)  ■  z(n  -  k)}  =  RxxiO)  -  Y  h{k)  ■  Rxz{k) 
k=-oo  k=-oo 
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The  second  line  is  obtained  from  the  first  using  the  orthogonality  principle.  By  Paxseval’s  relation,  we  can  then 

write  5  =  JRja;(0)  -  /  Sxz'‘{e^‘^)Hopt{e^'^)^  =  f  5ii(e^“)  -  Substituting  with 

y— TT  J—n 

Sxz‘{e^'^)  =  5iy*(e'’")  =  G(e'’^)5i*(e'’^),  we  obtciin 

£  =  f  5,,(e>")(l  -  Hopt{enG{en)^  (2.5) 

We  note  that  the  previous  equation  (2.5)  holds  only  for  Hopt{e^^).  The  reason  is  the  use  of  the  orthogonality 
principle  in  the  derivation  of  (2.5).  To  obtain  £  only  as  a  function  of  the  prefilter  we  substitute  Hopt{e^^) 

into  (2.5): 


£{\G\,b)=  r 

J  —  n 


The  problem  now  reduces  to  finding  the  prefilter  G{e^^)  that  minimizes  £  as  given  in  (2.6).  Two  points  are  in 
order  : 

1.  Since  the  mean  square  error  expression  (2.6)  is  a  function  of  \G{e^^')f  only,  we  will  be  actually  seeking  an 
expression  for  the  squ£Lred  magnitude  response  of  the  prefilter  rather  than  G{e^^). 

2.  It  is  clear  from  (2.6)  that  trying  to  derive  an  optimum  analytical  expression  for  \G{e^^’)f  can  be  quite 
tedious.  Instead  of  attacking  the  problem  as  it  is,  the  idea  is  to  transform  the  above  unconstrained  integral 
(2.6)  into  another  integral  with  a  power  constraint  on  the  prefilter  output.  The  problem  then  becomes  more 
mathematically  tractable  and  a  closed  form  expression  for  \G{e^^')f  can  be  obtained.  It  remains  to  show 
that  the  solution  of  both  problems,  the  original  one  and  the  equivalent  one,  is  the  same.  This  is  done  in  the 
follovdng  claim. 

Theorem  2.2.  The  squared  magnitude  response  \Gopt{^^^')t  minimizes  5(|G|,6),  given  as  in  (2.6),  is  also 
the  solution  of  the  following  constrained  optimization  problem: 

c2“2&5^^(ei‘^)  du; 


mm 

|G(€i-)|2 


/' 

J  — TT 


5xi(e>-)|G(ei“’)l  +c2-2»  2n 


subject  to: 


/: 


Sxx{en\G{enf^  =  1 


(2.7) 


(2.8) 


Proof.  The  role  of  the  magnitude  response  of  the  prefilter  is  basically  two  fold  ;  It  affects  the  spectral  shape  of 
the  quantizer  input  signal  y{n)  and  it  changes  the  quantizer  input  variance  and  therefore  the  noise  vciriance. 
The  idea  is  to  insert  a  multiplier  a  directly  before  the  quantizer.  The  insertion  of  this  multiplier  affect  only  the 
variance  of  the  quantizer  input.  One  can  then  show  that  the  mean  square  error  at  the  output  of  this  new  system 
is  unaffected  by  this  multiplier.  This,  in  turn,  indicates  that  we  can  always  fix  the  variance  of  the  quantizer  input 
signal  y{n)  without  changing  the  solution  of  our  original  problem.  To  prove  the  argument  formally,  we  proceed  as 
follows:  define 


G  =  aG(€'^“^')  such  that 


/’ 

J  — TT 


Sxx{en\G'ie^nf^  =  1 


(2.9) 
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Hence, 


=  C2 


.-2b 


£s„(eJ“)|G(«<")ff  =  ic2-’*.  £s..(ei")|G'(.>“)|"f  =  ic2-“  =  ±„}  (2.10) 


where  cr^,  is  the  quantization  noise  variance  of  the  system  of  Fig.  2.2  and  is  equal  to  c-2"^^.  The  postfilter  H 

of  the  new  system  is  given  by  (2.3)  with  ct^,  and  G  (e^^)  replacing  (Xg  and  G(e^‘^)  respectively.  Substituting  with 

(Tp  as  in  (2.10)  and  with  G'(e^’^’)  as  in  (2.9),  it  is  easy  to  see  that 


ff’optien  =  ^Hopt(en 


(2.11). 


The  filtering  scheme  of  Fig.  2.2  can  be  redrawn  as  in  Fig.  2.3.  Following  the  same  type  of  reasoning  as  before, 
the  mean  square  error  of  the  scheme  of  Fig.  2.3  can  be  thus  expressed  as: 

s'  =  r  5„(e^‘^)(l  -  H\,t{e^nG'{en)~  (2-12) 

J  — TT 

Substituting  in  (2.12)  with  G' as  in  (2.9)  and  with  H’ opt{e^‘^)  as  in  (2.11),  it  is  clear  that  £'  =£.  ■ 

As  a  consequence  of  the  above  analysis,  the  white  noise  source  variance  is  independent  of  the  variance  of  the 

quantizer  input  ^^..The  mean  square  error  expression  can  be  written  as  : 

S  =  r  c2-^^S„(e^‘-)  du;  ^3. 

J-n  S„(e>“)|G(eJ“)|^  +  c2-2>  Ztt 


2.2.  The  Optimum  Prefilter 

The  goal  now  is  to  find  |G(e-'‘^)|^  that  minimizes  (2.13)  under  the  integral  constraint  (2.8).  Since  the  magnitude 
squared  response  is  always  a  non  negative  function  of  u,  the  optimum  minimizing  solution  we  seek  must  be  non 
negative.  This  implicit  condition  is  incorporated  in  the  optimization  problem  as  a  pointwise  inequality  constraint. 
The  next  theorem  gives  an  expression  for  the  optimum  magnitude  squared  response  of  the  prefilter. 

Theorem  2.3.  ThepreSlter  |Gopt(e^")f  that  minimizes  (2.12)  under  the  constraint  f  |Gopt(e-’")|^^  =  1  must 

J  —TV 


2 

have  a  magnitude  response  |Gopt(e'’‘^)|  in  the  following  form: 


\Gopiienf  = 


+  c2 


-2b 


c2 


-2b 


^  V  a;  €  [~7r,7r]  for  which  the  r.h.s  is  >  0 

(2.14) 


otherwise. 


Proof.  The  minimization  of  the  functional  (2.13)  under  the  integral  constraint  (2.8)  and  the  positivity  condition 
belongs  to  a  class  of  calculus  of  variation  problems  known  as  isoperimetric  problems  [18],  [19].  An  outline  of  the 
major  steps  of  the  proof  vdth  the  corresponding  equations  is  given  below.  For  more  details,  we  refer  the  reader  to 
appendix  A. 

y 
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Step  1.  Problem  set  up.  We  transform  the  above  constrained  problem  into  an  unconstrained  one  by  lumping 
the  integrand  of  (2.8)  to  the  integrand  of  (2.13)  by  a  parameter  A(w).  This  leads  to  the  following  equation  : 


-a 


c-2-2‘-5„(eJ‘^) 


+  A-5„(e^")|G(e>-)| 


lA  du 


(2.15) 


.5„(ei‘^)|G(ei-)fTc:2-=>^  '  "  ''  J  2n 

The  parameter  A(w)  takes  care  of  the  integral  constraint  (2.8)  independent  of  frequency.  We  can  therefore  treat 
A(w)  as  a  constant  A.  This  last  statement  can  be  indeed  proved  formally  [See  page  175  of  [20]].  The  optimum 
magnitude  response  we  seek  must  obviously  be  positive  over  all  frequencies.  To  incorporate  this  constraint  in  our 
problem,  we  introduce  an  unspecified  parameter  0{w)  and  consider  now  the  problem  of  minimizing: 

c-2-2».5,x(e>‘-)  .  „  ,  . . 


/:g 


+  A  .  5„(e^")|G(e^'‘^)r  +  /3(u;)|G(e^‘^)f )  ^ 


(2.16) 


.5„(eJ^)|G(eJ^)P  +  c-2-2» 

The  value  of  the  parameter  /3{iv)  is  set  in  a  way  that  assures  that  the  positivity  constraint  is  never  violated.  We 
note  that,  unlike  the  parameter  A,  /3(u;)  in  this  ceise  takes  care  of  a  pointwise  constraint.  It  must  therefore  be  a 
function  of  u. 

Step  2.  Necessary  conditions  for  an  extremum.  The  key  necessary  condition  for  a  calculus  of  variation 
problem  is  the  Euler-Lagrange  equation.  For  this  problem,  this  is  equivalent  to  requiring  |G(e-’")|  to  satisfy  the 
following  equation  at  all  frequencies  : 


■  d 


c-2 


-26 


Sxzje^^) 


+  A  •  5„(e^")|G(e^' 


(2.17) 


d\G{ei^)f  V5„(e^-’)|G(eJ«‘')l"  +  c  •  2-2^ 

Solving  the  above  equation  leads  to  expression  (2.14). 

Step  3.  Sufficient  condition  for  an  extremum.  The  derivation  in  step  2  indicates  that  any  minimizing 
curv^e  for  (2.13)  under  the  integral  constraint  (2.8)  and  the  implicit  positivity  constraint  must  have  a  magnitude 
response  (2.14).  Using  the  convexity  of  functionals,  we  finally  prove  that  the  solution  (2.14)  is  not  only  necessary 
but  also  sufficient  for  a  minimizing  extremum.  ■ 

It  follows  immediately  from  this  last  theorem  that  the  optimum  prefilter  Gopt{^^^)  is  not  unique  since  the  phaise 
information  is  not  specified.  This  is  not  the  case  for  the  optimum  postfilter  Hopt{^^^)-  We  also  note  that  whenever 
\Gopt{^^^W  =  equation  (2.2)  simplifies  to  Hopt{e^^)  =  0  as  well. 


III.  FURTHER  ANALYSIS  OF  THE  OPTIMUM  ONE-CHANNEL  SYSTEM 
3.1.  The  Coding  Gain  Expression 

Assume  that  we  quantize  x{n)  directly  with  b  bits.  We  denote  the  corresponding  mean  square  error  (m.s.e)  by 
^direct-  We  then  use  the  optimum  pre  and  post  filters  around  the  quantizer.  With  the  rate  of  the  quantizer  fixed 
to  the  same  value  6,  we  denote  the  m.s.e  in  this  case  by  Enew  The  ratio  Gopt  =  ^direct/^new  is  called  the  coding 
gain  of  the  new  system  and,  as  the  name  suggests,  is  a  measure  of  the  benefits  provided  by  the  pre/post  filtering 
operation.  The  coding  gain  expression  for  the  system  of  Fig.  2.1  with  the  optimum  prefilter  Gopti^^^)  is  given  in 
the  following  theorem. 


7 


Theorem  3.1.  With  the  optimal  choice  ofpre-  and  postGlters,  the  coding  gain  expression  for  the  scheme  of  Fig. 


6opt  =  (l  +  c-2-^'’)-ghw  (3-1) 

as  long  as  the  right  hand  side  of  (2.14)  is  non-negative  V  u.  Here  Gkw  is  the  coding  gain  of  the  half  whitening 


scheme  and  is  given  by 


Proof.  Following  the  above  definition,  the  coding  gain  of  the  system  of  Fig.  2.1  can  be  expressed  as: 


Gopt  — 


c-2-2‘. 


r _ S^xjei^) _ ^ 

i-x  (5„(ei‘")|G(eJ>‘')l^  +  c  •  2-2^)  2rr 


Assuming  that  the  right  hand  side  of  (2.14)  is  always  positive,  firom  (2.14),  one  can  then  write. 


lG'(eJ“’)l^5xx(e^‘^)  +  c  •  2-2^  = 


(l  +  c-2-2‘’)  Vg^^(ei‘^) 


Substituting  (3.4)  into  (3.3),  we  can  simplify  the  coding  gain  expression  to  obtain: 


Gopt  =  {i+c-2-^>’) 


which  is  simply  the  half  whitening  gain  Qhw  (3-2)  multiplying  (1  +  c  •  2  ■ 

In  the  case  where  the  right  hand  side  of  (2.14)  is  negative  over  a  specific  interval  and  accordingly,  \G{e^  )|  is  set 
to  zero,  we  obtain  the  following  coding  gain  expression  : 


r>^2b\  hr: 


Gopt  =  (1  +  c  2-^^) . 


where  Q  is  the  set  of  frequencies  over  which  the  right  hamd  side  of  (2.14)  >  0.  We  still  expect  the  filtering 
scheme  under  study  to  outperform  the  half  whitening  scheme  in  this  case  but  it  is  not  clear  how  one  can  compare 

analytically  Gopt  to  Ghw 

Example  3.1  White  input  still  produces  gain.  In  this  example,  we  assume  that  the  input  x{n)  is  a  white  process 
with  variance  equal  to  one.  It  can  be  verified  for  this  case  that,  \G{ei^)f  =  1  V  w.  This  is  consistent  with  our 
earlier  observation  about  the  prefilter,  namely  that  it  exploits  the  spectral  shape  of  the  input.  The  post  filter 
is  a  constant,  independent  of  frequency.  The  coding  gain  of  the  half  whitening  scheme  is  one  since  it 
depends  only  on  the  spectral  shape  of  the  input.  However,  the  more  general  system  still  produces  a  coding  gain 
(H-C‘2”^^).  The  gain  results  from  the  “Wiener  filter  part”  of  the  postfilter  and,  consequently  ,  from  the  resulting 
prefilter  expression. 
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Remarks  on  The  Coding  Gain  Expression 

1.  The  coding  gain  expression  for  low  bit  rates.  It  is  quite  clear  from  theorem  3.1  that  the  system  of  Fig. 
1.1  will  eJways  outperform  the  half  whitening  scheme  as  long  cis  the  right  hand  side  in  equation  (2.14)  remains  non 
negative  for  all  frequencies.  The  difference  in  performance  is  basically  a  function  of  the  probabilistic  distribution  of 
the  quantizer  input  and  more  important  of  the  bit  rate.  The  lower  the  bit  rate  the  higher  the  coding  gain  will  be. 
However,  the  problem  is  that  as  we  quantize  at  lower  bit  rates,  the  quantizer  assumptions  made  at  the  beginning 
of  this  section  fail  and  all  the  previous  aneilysis  is  not  valid  anymore.  We  have  performed  some  side  simulations  to 
explore  the  validity  of  our  quantizer  model  at  various  bit  rates.  The  results  show  that  the  quantizer  model  used 
throughout  this  paper  is  excellent  at  6  >  4,  good  at  6  =  3  and  not  adequate  at  6  <  2. 

2.  The  coding  gain  expression  for  high  bit  rates.  By  letting  b  go  to  infinity,  one  can  easily  check  that  the 
right  hand  side  of  equation  (2.14)  becomes 

^  f  ^  _ ^  V  cj  G  [“7r,7r] 

and  is  positive  V  w.  Therefore,  the  coding  gain  expression  derived  in  theorem  3.1  can  be  used  and  as  6  go  to  infinity, 
Gopt  becomes  equal  to  Ghw-  At  high  bit  rate,  there  is  no  loss  of  generality  in  using  the  half  whitening  scheme.  The 
same  observation  was  first  mentioned  by  Goodman  and  Drouilhet  [21].  Although  the  final  conclusion  is  the  same, 
there  are  main  differences  between  their  work  and  ours.  First,  Goodman  and  Drouillet  did  not  derive  any  coding 
gain  expression.  It  was  quantitatively  unclear  how  much  we  can  benefit  from  using  the  more  general  system  of  Fig. 
2.1.  Second,  whereas  our  system  is  a  discrete  time  system,  the  system  analyzed  in  [21]  was  continuous  time  pre  and 
post-filters  sandwiching  a  sampler  and  a  quantizer.  Moreover,  Goodman  and  Drouillet  assumed  an  additive  white 
noise  source  model  for  the  quantizer  where  the  noise  source  is  uniform  and  independent  of  the  quantizer  input  and 
its  statistics.  Although  this  model  is  a  valid  one,  we  prefer  to  use  the  different  noise  model  proposed  in  [10]  by 
imposing  the  constraint  (2.1)  in  the  beginning  of  our  study.  Finally,  Goodman  and  Drouillet  replaced  the  sampler 
and  the  quantizer  by  an  additive  independent  noise  source.  By  doing  so,  the  system  becomes  identical  to  the 
communication  system  analyzed  by  Costas  [12].  The  starting  point  of  Goodman  and  Drouillet 's  correspondence 
is  therefore  Costas  result.  This  is  a  different  problem  as  we  pointed  out  in  the  introduction  of  this  paper.  In  our 
case,  we  cannot  use  Costas  result  directly.  The  use  theorem  2.2  is  essential  in  our  derivation  and  it  is  because  of 
this  theorem  that  the  quantization  problem  under  study  becomes  similar  to  a  communication  problem. 


3,2.  Analysis  under  a  Colored  Quantization  Noise  Assumption 

The  previous  analysis  can  be  repeated  assuming  that  the  quantization  noise  is  now  colored.  The  noise  power 
spectral  density  Sqqie^"^)  becomes  a  function  of  frequency.  The  remaining  assumptions  are  kept  the  same.  The 
optimum  postfilter  in  this  case  can  be  easily  rederived  and  is  given  by; 

1  5,,(e>") 


Hoptien  = 


(3,5) 
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The  corresponding  mean  square  error  expression  can  be  found  to  be: 


=  /' 

J  —TV 


djj 


(3.6) 


5**(eJ‘")|G(eJ“')l^  +  5,g(eJ“)  27r 
We  can  again  argue  that  the  mean  square  error  at  the  output  of  the  system  does  not  change  by  inserting  a 
multiplier  before  the  quantizer.  The  same  type  of  analysis  can  therefore  be  carried  out  producing  the  following 
expression  for  the  magnitude  response  of  the  optimum  prefilter: 


{  ^  .  (  ^~^Sqq{e^  ) - Sqqje^  )  ^  ^  ^  f-7r,7rl  for  which  the  r.h.s  >  0. 

=  J  \fl^VSxx{e^‘^)^  y/S^A^J 


otherwise. 
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IV.  OPTIMUM  PRE  AND  POST  FILTERING  WITH  FIRST  ORDER  FILTERS 

In  this  section,  we  will  constrain  and  G(eJ^)  to  be  first  order  causal  filters  with  monic  polynomials. 

First  order  filters  can  provide  a  substantial  boost  in  performance,  are  easy  to  track  mathematically  and  are  very 
economic  to  implement.  The  quantization  system  in  Fig.  1.1  is  still  a  uniform  quantizer.  We  will  again  jointly 
optimize  the  first  order  pre-  and  post  filters  to  minimize  the  m.s.e  under  the  constraint  (2.1).  All  the  other 
assumptions  of  section  II  are  the  same  as  before.  We  will  consider  two  main  cases:  a)  an  FIR  prefilter  with  an  HR 
postfilter  and  b)  an  HR  prefilter  with  an  FIR  postfilter. 


the 


4. 1.  The  FIR  prefilter  -  UR  postfilter  case 

In  this  subsection,  the  prefilter  is  in  the  form  1  -  az-K  The  postfilter  takes  the  form  Under 

constraint  (2.1),  the  mean  square  error  expression  is  derived.  It  is  a  function  of  two  variables  a  and  7  and  the  goal 
is  to  jointly  optimize  these  coefficients  to  minimize  the  mean  square  error.  The  next  theorem  gives  the  expression 
of  the  mean  square  error. 


Theorem  4.1.  Assume  that  the  prehher  G{e^^)  is  1  —  ae  and  that  the  postGlter  is  ^  ^ 

mean  square  error  as  a  function  of  a  and  7,  under  the  constraint  (2.1),  is  given  by: 

C(a,y)  =  e.2-“ 


Proof.  See  appendix  B. 

By  using  =  c2-^^crl  where  =  (1  +  a^)  ■  -  2afl„(l),  the  mean  square  error  expression  of  theorem  4.1 

can  be  rewritten  as  follows: 


£(a,7) 


(1 


-7^) 


(q  -  7)^ 

(1_72) 


•  7”*  Rxx{m) 

m=0 


(4.2) 


The  first  term  in  (4.2)  disappears  when  we  do  not  quantize  the  signal.  In  this  case,  the  mean  square  error  can  be 
reduced  to  zero  by  setting  q  equal  to  7,  i.e.,  the  postfilter  is  the  inverse  of  the  prefilter.  However,  in  the  presence 
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of  the  qugintizer,  the  choice  of  a  =  7  is  not  the  best  since  the  choice  of  7  affects  the  two  terras  in  (4.2)  in  different 
ways.  Equation  (4.2)  also  suggests  that,  at  high  bit  rate,  the  contribution  of  the  first  term  in  the  equation  will  be 
almost  negligible  compared  to  the  second  terra.  Hence,  as  b  increases,  we  should  expect  the  optimum  coefficients 
aopt  and  jopt  to  numerically  approach  each  other. 

Even  in  this  very  simple  case,  the  problem  is  highly  non-linear  in  the  filter  coefficients  a  and  7.  Closed  form 
expressions  for  the  coefficients  of  the  filters  in  terms  of  only  the  second-order  statistics  of  the  signal  cannot  be 
obtained.  However,  minimization  of  the  mean  square  error  can  be  done  numerically  using  for  example  MATLAB’s 
optimization  toolbox. 


4.2.  The  HR  pre  filter  -  FIR  postfilter  case 


We  can  easily  derive,  from  equation  (4.2),  the  mean  square  error  for  the  dual  case,  namely  when  the  prefilter  is 

f  _  jz~'^  postfilter  is  1  —  To  see  this,  aissume  first  that  there  is  no  quantization.  It  is  then  clear 

that  the  second  term  in  (4.2),  the  error  due  to  the  mismatch  of  the  coefficients,  will  not  change  by  switching  the 

position  of  the  filters.  When  quantization  is  present,  the  noise  term  becomes  (1  -h  a^)  (t^  where  the  noise  variance 

1  °° 

7”^  RxxifT^)  The  mean  square  error  expression  is  therefore  given  by  : 


m=0 


>1  /  \2  00 

€{an)  =  (1  +  a2)  c2-2‘  •  — ^  •  £  7"*  RM  +  7,  ~  J  •  ^  t”*  R^m) 


1  -  i 


m=0 


(1-7^) 


(4.3) 


m=0 


4.3.  The  Special  Case  of  First  Order  Filters  with  equal  coefficients 

A.  The  FIR  prefilter  -  HR  postfilter  case. 

When  Q  is  equal  to  7,  the  mean  square  error  becomes  a  function  of  one  parameter  a.  The  coding  gain  can  be  then 
expressed  as  follows; 

fi..(0)-(l  -  a\pt) 


Qopt  — 


(1  +  '  Rxx(0)  2a:gpt  ■  iixx(^) 


(4.4) 


If  Jixx(l)  =  0,  then,  the  above  coding  gain  expression  becomes  7^^ - ■  It  is  then  quite  clear  that  the 

(1  +  a-‘opt) 

optimum  coefficient  Oopt  is  equal  to  zero.  No  pre-and  post  filtering  can  enhance  the  reconstructed  output  and  the 
coding  gain  is  simply  unity.  On  the  other  hand,  If  R„(l)  =  i?ii(0),  then,  the  coding  gain  expression  becomes 
(1  -  approaches  1,  the  coding  gain  grows  unbounded.  The  tradeoff  is  the  stability  of  the  inverse 

filter. 

Having  taken  care  of  these  two  extreme  cases,  we  now  assume  that  0  <  |Rii(l)|  <  Exx(O)  £ind  introduce 

the  following  notation:  =  p  where  —  1  <  p  <  1.  The  problem  expressed  in  this  form  was  considered  by 

rixx 

Jayant  and  Noll  [10].  We  will  therefore  only  give  their  final  results. 

1.  The  optimum  coefficient  aopt  that  minimizes  the  mean  square  error  expression  is  given  by: 


O-opt 


=  1.^1  _  v/l  -  if  -  1  <  p  <  1 


(4.5) 
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(4.6) 


2.  The  coding  gain  expression  as  a  function  of  p  can  be  found  to  be 


Qopt  — 


1 


We  note  that  the  coding  gain  expression  (4.6)  is  also  the  coding  gain  of  a  2-channel  Karhiuien-Loeve  transform 
(KLT)  under  the  assumption  of  optimum  bit  allocation.  This  is  then  a  case  of  a  one  channel  biorthogonal  FB  that 
is  as  good  as  a  2  x  2  KLT  [an  example  of  a  two  channel  orthonormal  filter  bank]. 

B.  The  HR  prefilter  -  FIR  postfilter  case. 

When  a  is  equal  to  7,  the  mean  square  error  is  then  given  by 

(1  -H  a^)  c2-2‘  .  ■  X;  o'"  R.x(Tn)  (4-7) 

m=0 

In  this  case,  the  problem  is  highly  non-linear  in  the  filter  coefficient  q  and  an  analytical  solution  is  difficult 
to  obtain.  On  the  other  hand,  the  minimization  of  the  mean  square  error  can  be  easily  done  numerically.  Results 
are  illustrated  in  the  next  subsection  for  some  specific  examples. 


4.4.  Examples  of  Optimum  Filters  for  Specific  irtputs 

The  examples  given  in  this  subsection  correspond  respectively  to  the  cases  of  a  MA(1),  an  AR(1)  and  an  AR(5) 
input  process  x(n).  In  each  case,  we  compare  the  coding  gain  of  the  general  first  order  system  [q  7]  to  the 
coding  gain  obtained  from  a  first  order  system  with  q  equal  to  7.  The  optimization  of  the  coefficients  is  done 
numerically  using  MATLAB’s  optimization  toolbox  whenever  an  analytical  expression  is  difficult  to  obtain.  For 
the  first  two  examples,  we  also  include  in  our  comparison  the  corresponding  coding  gain  of  a  half  whitening  scheme 
for  the  FIR/IIR  cases  [the  coding  gain  of  a  half  whitening  scheme  establishes  a  theoretical  bound  on  the  coding 
gain  of  the  first  order  one  channel  biorthogonal  system]. 

Example  4.3.1  Cast  of  a  MA(1)  process.  Assume  that  the  input  x{n)  is  a  zero  mean  MA(1)  process  with  an 
autocorrelation  sequence  in  the  form 

{1  A:  =  0. 

0  otherwise. 


R  f  1) 

It  is  well  knowm  that  a  MA(1)  process  has  to  have  ^  1/2  ^  ensure  that  the  power  spectral  density  is 

indeed  non  negative.  We  therefore  restrict  6  to  be  between  —1  and  1.  For  the  FIR  prefilter  -  HR  postfilter  case, 
when  a  is  equal  to  7,  the  ratio  Rxx(l)/-Rzx(0)  is  now  equal  to  0/(1  -1-  e^).  We  therefore  simply  replace  p  in 
equations  (4.5)  and  (4.6)  by  0/(1  +  0^)  to  obtain  expressions  for  the  optimum  coefficient  Qopt  and  the  optimum 
coding  gain  Qopt-  The  power  spectrum  of  the  MA(1)  process  is  given  by; 


Sp,{en 


1  -  2 


0 


(1  02) 


cos{uj) 


(4.8) 
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Substituting  (4.8)  in  (3.2),  the  coding  gain  expression  of  the  half  whitening  scheme  for  a  MA(1)  process  is  given 

Cw  = - (<-9) 


(J  V(i  +  9*  -  2ecM{u}}  ^) 

The  integral  in  (4.9)  is  equal  to  F(-0.5  ,  -0.5  ;  1  ;  0^)  where  F(o  ,  6  ;  c  ;  d)  is  Gauss’s  hypergeometric  function. 
From  [22],  F(— 0.5  ,  —0.5  ;  \  ’,0^)  can  be  rewritten  as  (1  +  0)  •  F(— 0.5  ,  0.5  ;  1  ;  40/(1  +  0)  ).  This,  in  turn, 
can  be  simplified  to  (1  +  0)  •  -  •  F(2^/(i^/(l  +  0))  where  F(.)  is  the  complete  elliptic  integral  of  the  second 

TT 

kind. 

The  optimization  of  the  coefficients  for  the  HR  prefilter-FIR  postfilter  case  and  the  FIR  prefilter-IIR  postfilter 
with  a  7^  7  were  all  done  numerically  using  MATLAB’s  optimization  toolbox  routine  “fmins.m”.  The  plots  of  the 
coding  gain  for  the  FIR/IIR  case  are  illustrated  in  Fig.  4.1,  Fig.  4.2  and  Fig.  4.3  as  the  bit  rate  h  varies  from  1  to 
3.  Similarly,  the  plots  of  the  coding  gain  for  the  IIR/FIR  case  are  shown  in  Fig.  4.4,  Fig.  4.5  and  Fig.  4.6  also  as 
h  varies  from  1  to  3.  FVom  these  figures,  we  can  observe  that  as  the  bit  rate  increases,  there  is  no  loss  of  generality 
in  assuming  a  to  be  equal  to  7.  We  also  note  that  the  coding  gain  obtained  in  the  FIR/IIR  case  is  higher  than 
the  dual  case  for  the  same  process  and  bit  rate.  This  is  primarily  due  to  the  fact  that  the  optimum  coefficients  in 
the  IIR/FIR  case  are  numerically  close  to  zero  and  the  coding  gain  is  therefore  close  to  one. 

Example  4.3.2.  Case  of  an  AR(1)  process.  Assume  that  the  input  x{n)  is  a  zero  mean  AR(1)  process  with  an 
autocorrelation  sequence  in  the  form  Fji(A:)  =  where  p  must  be  between  -1  and  1. 

For  the  FIR  prefilter  -  HR  postfilter  case,  when  q  is  equal  to  7,  the  ratio  RzxiX) ! Rxx{^)  is  equal  to  p.  Qopt  is 
therefore  given  by  (4.5)  and  the  coding  gain  Gopt  is  given  by  (4.6).  The  power  spectrum  of  the  AR(1)  process  is 

1  - 


Sxxien  -  1  ^  _  2pcos{uj) 

Substituting  (4.10)  in  (3.2),  the  half  whitening  coding  gain  expression  for  the  AR(1)  process  is  as  follows; 

1 


Qhw  — 


1 


du)\ 


(4.10) 


(4.11) 


^(1  +  —  2pCOS{u}))  277  / 

The  integral  in  (4.11)  is  equal  to  ^  •  K{p)  where  K{p)  is  the  complete  elliptic  integral  of  the  first  kind  [22]. 

The  optimization  of  the  coefficients  for  the  HR  prefilter-FIR  postfilter  case  and  the  FIR  prefilter-IIR  postfilter 
with  Q  7^  7  were  all  again  done  numerically  using  MATLAB’s  optimization  toolbox  routine  finins.m  .  The  plots 
of  the  coding  gain  for  the  FIR/IIR  case  are  illustrated  in  Fig.  4.7,  Fig.  4.8  and  Fig.  4.9  as  the  bit  rate  b  varies 
from  1  to  3.  Similarly,  the  plots  of  the  coding  gain  for  the  IIR/FIR  case  are  shown  in  Fig.  4.10,  Fig.  4.11  and 
Fig.  4.12  also  as  b  varies  from  1  to  3.  The  observations  mentioned  at  the  end  of  the  previous  MA(1)  example  still 
hold  for  this  caise. 

Example  4.3.3  Case  of  an  AR(5)  process.  The  autocorrelation  function  of  such  a  process  extends  to  infinity 
and  doesn’t  have  a  simple  closed  form  expression.  The  main  problem  is  the  infinite  summation  in  the  form 
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£  7'"  Rxx{m)  found  in  equations  (4.2),  (4.3)  with  a  7  and  (4.7)  with  7  =  0.  Our  approach  is  to  truncate 
tlds  infinite  summation  with  the  assumption  that  after  a  certain  lag  m,  the  correlation  coefficients  axe  negligible. 
For  this  AR(5)  process,  we  set  R(0)  =  1,  R{1)  =  0.86,  R{2)  =  0.64,  R(3)  =  0.4,  J?(4)  =  0.26,  H(5)  =  0.2  and 
=  0  V  m  >  6.  The  values  of  the  correlation  coefficients  are  obtained  fi-om  page  37  in  [10].  Table  1 
summarizes  our  coding  gain  results  in  db  for  the  different  cases  and  bit  rates.  Again,  as  6  mcreases,  we  observe 
that  there  is  almost  no  loss  in  coding  gain  if  we  assume  that  a  =  7-  We  also  observe  that,  at  low  bit  rate, 
e.g.  6  =  1,  the  coding  gain  of  the  more  general  system  is  very  small.  This  suggests  that  the  gain  obtained  from 
searching  over  a  more  general  class  than  the  biorthogonal  class  may  not  be  worth  the  added  complexity.  Fmally, 
as  was  the  case  for  the  two  previous  examples,  the  FIR/IIR  scheme  outperforms  substantially  the  dual  case. 

V.  REPLACING  THE  QUANTIZER  SYSTEM  BY  AN  ORTHONORMAL  UNIFORM  PRFB 

Consider  the  M  channel  maximally  decimated  uniform  SBC  of  Fig.  1.2.  The  boxes  labeled  Q  are  modeled 
by  additive  noise  sources  in  the  manner  described  in  the  introduction.  Throughout  this  section,  we  will  assume 
that  the  subband  quantization  noise  sources  are  white  and  pairwise  uncorrelated.  If  we  interpret  this  FB  as  a 
sophisticated  quantizer,  the  use  of  pre-  and  post  filters  around  the  FB  can  very  well  increase  the  coding  gain.  In  a 
recent  paper,  Djokovic  and  Vaidyanathan  [16]  analyzed  the  system  of  Fig.  1.1  where  the  quantization  system  QS 
is  a  uniform  orthonormal  PRFB  and  the  postfilter  is  the  inverse  of  the  prefilter.  The  authors  gave  a  formula  for 
the  optimum  allocation  of  bits  in  the  subbands.  Furthermore,  they  showed  that  minimizing  the  mean  square  error 
of  the  so  called  prefiltered  paraunitary  (PPU)  PRFB  can  be  done  by  separately  optimizing  the  pre/post  filtering 
scheme  and  the  orthonormal  filter  bank.  Their  proposed  solution  was  a  half  whitening  scheme  sandwichmg  an 
optimum  orthonormal  PRFB.  A  generalization  of  the  scheme  of  Djokovic  and  Vaidyanathan  would  be  again  to 
relax  the  assumption  that  the  postfilter  is  the  inverse  of  the  prefilter.  An  analytical  optimum  solution,  if  it  exists, 
must  incorporate  the  joint  optimization  of  the  orthonormal  PRFB  and  the  pre-  and  post  filters.  It  is  not  clear  that 
a  separate  optimization  of  the  pre-  and  post  filters  and  the  orthonormal  PRFB  still  holds  in  this  case.  Furthermore, 
any  developed  optimum  bit  allocation  formula  must  include  the  pre-and  postfiltering  operation. 

In  the  remainder  of  this  section,  we  will  not  attempt  to  solve  the  problem  described  above.  We  will  instead 
provide  a  suboptimum  procedure  that  relies  on  the  results  derived  in  section  II.  We  will  see  that  even  in  this  simpler 
case,  two  theorems  must  be  first  established.  The  first  step  in  the  procedure  is  to  optimize  the  orthonormal  uniform 
PRFB  for  a  certain  WSS  input  i(n).  Several  authors  have  recently  shown  [6],  [8],[9]  that  the  optimum  orthonormal 
uniform  PRFB,  the  one  that  maximizes  the  coding  gain  as  defined  in  section  III,  will  consist  of  antialias  filters. 
A  discrete  time  filter  is  said  to  be  an  antialias(M)  filter  if  its  output  can  be  decimated  M-fold  without  aliasing. 
Since  this  requires  infinite  attenuation  in  the  stopbands,  anti-alias  filters  are  therefore  a  class  of  ideal  filters.  The 
second  step  in  the  procedure  is  to  perform  the  optimum  bit  allocation  operation  in  the  usual  way  [23].  After 
optimally  allocating  the  bits,  we  would  like  to  apply  the  pre-  and  post  filters  derived  previously  in  section  II. 


14 


In  order  to  do  this,  we  need  first  to  replace  the  whole  optimum  orthonormal  PRFB  by  an  additive  noise  source, 
say  v(n).  This  noise  source  v(n)  must  be  WSS  and  uncorrelated  with  the  prefilter  output  j/(n).  Second,  the 
variance  of  the  input  process  must  be  related  to  the  PRFB  output  noise  variance  Ssbc  hi  &  similar  fashion 
as  in  equation  (2.1).  A  major  problem  is  the  following:  In  the  presence  of  quantizers,  it  is  well  known  that  the 
output  of  a  uniform /non  uniform  PRFB  is  in  general  a  cyclo-wide  sense  stationary  (CWSS)  process.  The  cyclo 
wide  sense  stationarity  is  due  to  the  passage  of  the  quantization  noise  through  the  interpolators  [24].  We  provide 
two  results  describing  important  cases  that  guarantee  the  wide  sense  stationarity  of  the  quantization  noise  of  a 
uniform  orthonormal  PRFB.  Since  the  results  hold  for  the  non  uniform  decimation  case,  the  proofs  will  assume 
a  non  uniform  maximally  decimated  orthonormal  PRFB  case.  A  non  uniform  SBC,  shown  in  Fig.  5.1,  is  a  SBC 
with  unequal  subband  decimation  ratios  n*.  The  boxes  labeled  Q  represent,  as  before,  uniform  quantizers  that 
are  modeled  by  additive  noise  sources. 

Theorem  5.1.  Under  optimum  bit  allocation,  the  output  noise  of  a  [possibly  non  uniform]  orthonormal  PRFB  is 
WSS  provided  the  subband  quantization  noise  sources  are  White,  Uncorrelated  and  Zero  Mean  (WUZE  assump¬ 
tions). 

Proof.  The  proof  is  now  established  through  the  following  series  of  steps: 

1.  Soman  and  Vaidyanathan  [23]  showed  that  for  a  non  uniform  orthonormal  PRFB,  the  variances  of  the  subband 
quantization  noises  should  be  equal  under  optimum  bit  allocation.  Because  we  are  assuming  optimum  bit  allocation 
in  our  theorem,  we  can  immediately  conclude  that  the  noise  variances  in  the  non  uniform  orthonormal  PRFB  should 
be  equal  to  each  other. 

2.  It  is  well  known  [25], [26], [27]  that  an  M  channel  non  uniform  FB  can  be  redrawn  as  an  L  channel  maximally 
decimated  uniform  FB  where  L  =  nkPk-  The  M  set  of  analysis  and  synthesis  filters  {H*(z),F*(z)}  are  replaced 
by  the  set  of  L  filters  {H'^{z),fI{z)}  in  the  uniform  system  where  L  >  M.  The  main  goal  at  this  point  is  to 
develop  the  form  of  the  power  spectral  density  matrix  of  the  subband  quantization  noise  Sqq(e-’^)  in  the  equivalent 
L  channel  maximally  decimated  uniform  FB.  We  first  observe  that  the  white  noise  assumption  guarantees  that, 
for  the  kth  channel,  the  quantization  noises  in  its  corresponding  pk  channels  are  uncorrelated.  Furthermore, 
the  variance  of  the  qu^lntization  noise  is  the  same  in  all  the  pk  channels.  Combining  this  observation  with  the 
conclusion  of  step  1,  it  is  easy  to  see  that  Sqq{e-'")  should  be  equal  to  cr^I  where  Sqq(e-’’“)  is  an  L  x  L  matrix. 

3.  Since  the  non  uniform  maximally  decimated  FB  is  orthonormal  and  exhibits  the  perfect  reconstruction  (PR) 
property,  then,  it  follows  that  the  analysis  polyphase  matrix  E’(eJ“)  of  the  equivalent  L  channel  uniform  FB  is 
lossless  ,  i.e.,  E'  (eJ‘^)E'^(e-’‘^)  =  I  (orthonormality)  and  the  synthesis  polyphase  matrix  R'(e^'")  of  the  equivalent 
L  channel  uniform  FB  is  equal  to  E'^e^'*^)  (PR  property)  [26].  The  power  spectral  density  matrix  of  the  output 
quantization  noise  Sw(e-’“)  is  equal  to  R'(e-’“)Sqq(e-’")R'\e-’")  which  can  be  evaluated  as  crp  using  the  above 
properties.  This  means  that  the  output  noise  v{n)  is  an  interleaved  version  of  L  uncorrelated  white  noise  sources. 
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each  of  variance  (Tg.  So,  v(n)  itself  is  white  with  variance  (7g.  ■ 

Since  the  above  theorem  holds  for  a  non  uniform  maximally  decimated  orthonormal  PRFB,  it  includes  the  uniform 
decimation  case.  The  output  quantization  noise  v(n)  in  theorem  5.1  is  white  with  variance  <Tg.  Furthermore,  v(n) 
and  y(n)  are  uncorrelated.  The  problem  with  the  optimum  bit  allocation  is  that  it  yields  non  integer  solution  for 
the  bits.  If  we  use  a  simple  rounding  procedure  or  a  more  sophisticated  algorithm  [28]  to  obtain  integer  solutions, 
the  assumption  of  equal  quantizer  noise  variances  is  not  valid  any  more.  Nevertheless,  in  the  next  theorem,  we 
prove  that  even  with  the  more  practiced  assumption  of  different  quantization  noise  variances,  the  output  of  a 
non-uniform  orthonormaJ  PRFB  with  antialias  filters  will  be  wide  sense  stationary. 

Theorem  5.2.  The  output  noise  of  a  [possibly  nonuniform]  orthonormaJ  PRFB  consisting  of  antialias  Glters  is 
WSS  provided  the  subband  quantization  noise  sources  are  zero  mean  and  pairwise  uncorrelated. 

Proof.  Consider  the  s3mthesis  bank  of  a  non  uniform  PRFB.  The  quantization  noise  sources  qk{n)  at  the  input 
of  the  interpolators  are  assumed  to  be  individually  WSS  with  power  spectrum  Sqi^{e^^)  and  are  pairwise  uncorre¬ 
lated.  Since  the  filters  Fkie^^^)  are  antialias  for  all  k,  then,  the  upsampled  and  filtered  noise  sequences  Vk{n)  are 
individually  WSS  [24].  Furthermore,  since  the  interpolated  noise  sequences  Vk{n)  are  linear  combinations  of  the 
input  noise  sources  qk{n),  the  uncorrelatedness  property  is  preserved.  This  can  be  verified  by  writing  the  output 
vector  v(n)  as  a  time  varying  linear  combination  of  the  vector  q(n)  and  taking  expectations.  The  interpolated 
noise  sources  Vk{n)  are  therefore  jointly  wide  sense  stationary  which  implies  that  their  sum  v{n)  is  WSS.  ■ 

We  emphasize  the  fact  that  neither  the  whiteness  of  the  noise  sources  nor  the  equal  variance  assumptions  are 
required  for  this  theorem  to  hold.  We  note  that  the  output  quantization  noise  v{n)  in  theorem  5.2  is  still  uncor¬ 
related  with  the  prefilter  output  y(n).  However,  in  this  case,  v{n)  is  not  white.  If  the  subband  quantization  noise 
sources  are  white,  it  is  easy  to  see  that  the  power  spectral  density  of  the  PRFB  output  noise  is  piecewise 

constant.  The  magnitude  of  each  piece  of  is  equal  to  (7g^  for  some  k.  The  location  of  the  constant  piece 

is  determined  by  the  passband  of  the  corresponding  synthesis  filter  Fk{eJ^).  The  variance  of  the  output  noise  v{n) 

1 

is  the  average  of  the  individual  noise  variances  —  ^  (Tg^ . 

k=0 

The  above  two  theorems  permits  the  continuation  of  our  suboptimum  procedure.  The  optimum  bit  allocation 

[•without  including  the  pre-  and  post  filters]  allows  us  to  relate  the  variance  of  the  input  process  to  the  FB 

£.2-2*’ 

output  noise  variance  £sbc  by  SsBC  =  The  optimum  orthonormal  FB  is  a  special  case  of  a  non 

ypu 

uniform  PRFB  with  antialias  filters  for  which  theorem  5.2  applies.  The  FB  can  be  therefore  modeled  as  a  WSS 
noise  source  that  is  uncorrelated  -with  the  prefilter  output  sequence  y(n)  and  has  a  variance  proportional  to  al- 
This  is  the  perfect  setting  for  our  previous  pre-  and  post  filtering  analysis.  The  complete  system  is  sho'wn  in  Fig. 
5.2.  The  expressions  for  the  optimum  postfilter  and  the  magnitude  response  of  the  optimum  prefilter  are  given 
respectively  by  (2.3)  and  (2.14)  if  the  noise  v{n)  is  white  [case  of  theorem  5.1]  or  by  (3.5)  and  (3.7)  if  the  noise 
v{n)  is  colored  [case  of  theorem  5.2].  For  either  cases,  the  coding  gain  of  the  system  of  Fig.  5.2  can  be  easily 
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obtained  as  (1  +  7; — )  GhwGpu  provided  the  right  hand  side  of  (2.14)  or  (3.5)  is  always  positive.  The  next  toy 
Gpu 

example  illustrates  the  above  procedure  and  provide  some  numerical  results. 

Example  5.1.  We  assume  that  the  input  x{n)  is  a  zero  mean,  real,  WSS  random  process  with  a  triangular  power 
spectral  density  Sxx{e^‘^)  as  shown  in  Fig.  5.3.  The  optimum  orthonormal  FB  in  this  case  is  the  well-known 
contiguous  ideal  brick  wall  FB  [6],  [7],  [8],  [9].  The  coding  gain  of  an  orthonormal  FB  after  the  optimum  allocation 
of  subband  bits  is  in  general  given  by  [23]: 


Gpu 


a 


2 

X 


(n 


Af-l 

ife=0 


where  is  the  variance  of  x{n)  and  is  the  variance  of  the  kth  subband  signal.  For  the  ideal  brick  wall  FB  and 
a  triangular  power  spectral  density,  the  above  coding  gain  expression  can  be  simplified  to  the  following  expression: 


Gpu  = 


0.5 


(nf=o'(l  -  {2k  +  l)/2M)f 


We  then  apply  the  optimum  pre-  and  post  filters  at  the  input  2ind  output  of  the  FB  respectively.  For  cui  average 
bit  rate  6  =  3,  the  constant  c  =  0.75  and  the  number  of  channels  M  =  2,  it  can  be  verified  that  the  optimum 
prefilter  [in  both  cases  of  white  and  colored  noise]  is  never  set  to  zero  at  any  frequency  and,  therefore,  we  can  use 

^-2b 

the  formula  Gopt  =  (1  +  7; — )  GhwGpu-  Using  the  above  data,  we  obtain  Gpu  =  and  Ghw  =  §•  Finally,  the 

Gpu 

theoretical  bound  on  the  coding  gain,  namely  the  prediction  gain,  is  given  by  [1]: 


Gth  = 


exp{Zjn{Sxx{e=-))t) 


For  this  case,  Gth  is  equal  to  Expressing  the  above  quantities  in  db,  we  get  :  Gpu  =  0.625  db,  Ghw  =  0.51  db, 
=  1.19  db  and  Gth  =  1*33  db.  It  is  important  to  observe  the  relative  gain  obtained  using  the  pre-  and  post 
filtering  operation  rather  than  the  absolute  increase  in  coding  gain.  Clearly,  we  get  a  substantial  increase  by  using 
the  pre-  and  post  filters  as  Gopt  approaches  Gth- 


VI.  CONCLUDING  REMARKS 

In  this  paper,  we  have  studied  pre-  and  post  filtering  around  a  quantization  system  QS  under  the  key 
assumption  that  the  quantization  noise  variance  is  proportional  to  the  variance  of  the  quantization  system 
input.  For  the  case  where  QS  is  a  uniform  quantizer,  we  provided  joint  optimum  solutions  for  the  ideal  pre-  and 
post  filters.  Using  these  solutions,  we  then  derived  a  coding  gain  expression  for  the  system  of  Fig.  1.1.  The 
importance  of  this  expression  is  that  it  clearly  indicates  that,  at  high  bit  rate,  there  is  no  substantial  loss  of 
coding  gain  if  we  set  the  postfilter  to  be  the  inverse  of  the  prefilter.  We  then  considered  two  cases  of  first  order 
pre-  and  post  filters:  FIR/IIR  and  IIR/FIR.  For  each  case,  we  obtained  a  mean  square  expression,  optimized 
the  coefficients  a  and  7  and  compared  coding  gain  performances  with  the  case  of  a  equal  to  7  through  a  set 
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of  examples.  Finally,  we  considered  the  case  where  the  quantization  system  QS  is  an  orthonormal  FB.  To  be 
able  to  apply  the  previously  optimized  pre-  and  post  filters  at  the  input  and  output  of  the  FB  respectively,  we 
developed  two  theorems  that  guaranteed  the  wide  sense  stationanty  of  the  filter  bank  output.  AVe  emphasize  again 
that  applying  the  pre-  and  post  filters  in  the  manner  described  in  section  V  is  suboptimum.  While  this  paper 
deals  with  some  of  the  current  issues  in  the  subband  coding  area,  it  opens  up  other  interesting  and  challenging 
problems.  One  example  that  quickly  comes  to  mind  is  the  extension  of  this  work  to  the  M  channel  case.  A  globally 
optimum  solution  should  include  a  strategy  for  the  allocation  of  the  subband  bits  as  well  as  a  joint  optimization 
of  the  analysis  and  synthesis  sections  of  the  SBC.  Another  problem  is  the  optiffiuTn  generalization  of  Djokovic  and 
Vaidyanathan’s  scheme.  If  fully  optimized,  the  more  general  scheme  of  Fig.  5.2  should  outperform  the  scheme 
proposed  by  Djokovic  and  Vaidyanathan.  An  easy  way  to  see  this  is  to  simply  put  a  Wiener  filter  at  the  output 
of  the  half  whitening  filter  sandwiching  an  optimum  orthonormal  PRFB.  Even  in  this  suboptimum  procedure,  the 
mean  square  reconstruction  error  cannot  increase.  Finally,  the  coding  gain  expression  (4.6)  for  a  first  order  FIR 
prefilter/  HR  postfilter  leads  to  another  interesting  question.  As  we  observed  before,  equation  (4.6)  is  also  the 
coding  gain  of  a  2-channel  Karhunen-Loeve  transform  under  the  assumption  of  optimum  bit  allocation.  This  is 
therefore  the  case  of  a  first  order  half  whitening  scheme  performing  as  good  as  a  2x2  KLT  [case  of  a  2-channel 
orthonormal  FB].  This  is  interesting  in  view  of  the  fact  that  the  asymptotic  coding  gain  of  a  KLT  is  higher  than 
that  of  a  half  whitening  filter.  A  natural  question  then  arises:  how  does  the  coding  gain  of  a  KLT  of  block  length 
(M  -h  1)  compare  to  the  coding  gain  of  a  half  whitening  scheme  of  order  M  ?  The  answer  to  this  question  remains 
open. 


Appendix  A. 

step  1.  We  have  argued  in  the  proof  of  theorem  2.3  that  the  parameter  A(w)  is  independent  of  frequency.  We 
proceed  to  prove  that  it  is  a  positive  constant.  Assume  for  the  moment  that  /3(w)  is  equal  to  zero  in  (2.17)  and 
denote  the  integrand  of  (2.17)  by  F  +  AW  where 

c  •  2-2‘ •  5xi(e^^ 


F  = 


5xx(eJ“)|t?(e^“’)l'  +  c-2-2» 


and  W  =5xx(e^‘^)|G(e^'") 


From  the  theorem  on  page  43  in  [18],  we  see  that  if  |G(e-’‘^)f  is  an  extremum  of  (2.17)  with  =  0  [but 

2 

is  not  in  the  same  time  an  extremum  of  W],  then,  there  exists  a  constant  parameter  A  such  that  dF/d\G{eP'^)\  + 
0W/9|G(e-’")|^  =  0  for  all  w.  Since  |G(e^’")|^  is  not  an  extremal  for  W,  then,  there  is  a  Wo  such  that 
SW/5|G(e^‘^)|V  Oatw  =  uio-  This  yields 


dFld\G{t^'^)\^ 

dWld\G{ei'^)t 


The  numerator  and  denominator  of  (A.l)  are  found  to  be: 

dF _ c-2-^^5xx(e^'^)" 

9|G(e-’“)|  (5xx(eJ")|G(eJ>^')l^  +  c '  2-^'') 


k 


dW 

d\G{ei'^)t 


(Al) 


(A.2) 
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Substituting  (A.2)  into  (A.l),  we  obtain  the  following: 

which  in  particular  shows  that  A  >  0. 

Step  2.  The  necessarj'  conditions  for  \G{e^^)f  to  be  a  minimum  of  (2.16)  are  sununaxized  next. 
1.  Define  G'  to  be  the  derivative  of  |Cj(e^“’)|^  with  respect  to  u.  The  Legendre  condition  [18], 


-^(F  +  AW")>0  Vw 
dG' 


(^.3) 


must  be  satisfied.  In  our  case,  this  condition  is  satisfied  trivially  because  neither  the  functional  (2.14)  nor  the 
constraint  (2.8)  are  functions  of  the  derivative  of  |G(e-^^’)|  . 

2.  must  satisfy  the  Euler-Lagrange  equation  for  the  functional  (2.16)  i.e  |G(e-^‘^)|^  must  satisfy  (2.17). 

The  Euler-Lagrange  equation  (2.17)  is  a  pointwise  relation  that  must  be  satisfied  at  all  firequencies.  The  value  of 
the  unknown  parameter  P{uj)  in  the  right  hand  side  is  therefore  set  according  to  two  criterions:  First,  the  choice 
of  should  not  violate  the  Euler-Lagrange  equation  at  any  frequency.  Second,  the  choice  of  /3(a;)  should  insure 
the  positivity  of  the  solution  at  all  frequencies.  There  are  therefore  two  possible  values  for  )3(cj). 

Case  of  /3(u?)  =  0.  Assume  first  that  ^{uS)  =  0.  The  left  hand  side  of  (2.17)  becomes  zero  and,  in  this 

case,  equation  (2.17)  can  be  interpreted  cts  the  Euler-Lagrange  equation  for  an  exactly  similar  problem  without  a 
positivity  constraint  on  the  solution.  Therefore,  for  those  frequencies  where  f3{u;)  =  0,  the  positivity  constraint  is 
actually  ineffective  and  the  solution  we  obtain  must  be  >  0  at  those  frequencies.  The  optimum  magnitude  squared 
repose  |Gopf  (e-^'^)f  in  this  case  is  determined  from  (2.17)  with  the  right  hand  side  set  to  zero.  Perform  the  partial 
differentiation  in  (2.17)  and  equating  the  result  to  zero,  the  following  equation  can  be  obtained: 

(^S^^{en\G{en\  +C'2-^»j  =  •  5xx(e^“)  (AA) 

Taking  the  square  root  of  (A.4)  and  simplifying,  we  get:  where  7  =  y/X. 


\G{e 


=  - 


.2-26 


c-2 


-26 


t(e-’'“)  5„(e>“) 


(A.b) 


where  7  =  y/X.  Substituting  |G(e-'‘^)f  as  in  (A. 5)  into  the  constraint  (2.8),  we  obtain: 

dijJ 


-J 

Hence,  the  constant  7  is  given  by: 


I  c-2-2fe 


“l  +  c-2-2^  ^  27r 


(A.6) 


(A.7) 
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Substituting  7  as  in  (A.7)  in  (A.5),  we  therefore  obtain  the  first  part  of  (2.14),  namely  that: 


|G(e^-)P 


1  /  1  +  c  •  2-2*’  _  c-2-2‘’  \ 


for  all  frequencies  for  which  the  right  hand  side  of  (A. 8)  is  non-negative. 


(A.8) 


Case  of  /3(fa>)  ^  0.  At  some  p£u'ticular  frequency,  the  solution  obtained  in  case  1  might  turn  out  to  be  negative. 
The  positivity  constraint  is  obviously  violated.  At  such  a  frequency,  /3(w)  should  not  bet  set  to  zero  anymore. 
Since  the  Euler-Lagrange  equation  must  be  satisfied  at  all  times,  we  must  set  ^{w)  to  be  equal  to: 


_ ( - c-2  - -  +  A-5„(e^“)|G(e^‘")|") 

d\G{e}^)f  V5„(eJ")|G(eJ«^’)f -l-c-2-2»  J 

The  sign  of  0iu)  in  this  case  is  important  to  make  sure  that  the  positivity  constraint  is  not  violated.  For  our 

problem,  should  be  non  positive.  Finally,  it  remains  to  find  the  value  of  |Gopt(e^  )|  when  ^(w)  ^  0.  The 

Euler-Lagrange  equation  cannot  be  used  anymore  because  it  determines  the  unknown  parameter  ^(w).  However, 

2 

we  can  simply  observe  that  \Gopt{^^^)\  cannot  be  greater  than  zero.  According  to  the  first  case,  if  |Gop<(c-^  )| 
is  set  to  any  value  greater  than  zero,  /3(u;)  should  be  zero.  The  only  possible  remaining  value  for  |Gopt(e-’‘^)|  is 
therefore  zero.  This  argument  establishes  the  second  part  of  (2.14). 

From  the  above  construction,  we  see  that  |Gopt(e'^‘^)|  is  a  smooth  function  of  w  (i.e  it  is  continuous  with 
continuous  first  order  derivative)  everywhere  except  at  the  frequencies  where  it  has  to  be  forced  to  zero  (so  it 
does  not  turn  negative).  The  frequencies  Uk  at  which  |Gop((e'*‘’’)|  is  set  to  zero  are  called  comer  points.  To  be 
an  acceptable  piecewise  smooth  solution,  |Gopf(e'*‘^)|*  must  satisfy  the  so  called  Weistrass-Erdmann  conditions  at 
those  frequencies.  In  our  case,  the  Weistrass-Erdmann  conditions  reduce  to  the  requirement  that  the  integrand  in 
(2.17)  be  a  continuous  function  of  u)  at  the  corner  frequencies  w*.  This  requirement  is  indeed  satisfied  because  the 
integraind  is  a  continuous  function  of  |G(e'^“)|^  which  in  turn  is  continuous  in  u  even  at  the  corner  points  w* 
Step  3.  We  would  like  now  to  prove  that  the  magnitude  response  expression  (2.14)  is  not  only  necessary  but  also 
sufficient  for  the  optimality  of  the  prefilter.  We  introduce  the  following  notation  by  rewriting  (2.17)  as  follows: 


where 


fi(w,y(w))  = 


■  S, 


r(e^‘") 


Sxz(e^‘^)  •  y(w)  + 
f2{u),y(u}))  =  A  •  5a;i(e^“)  •  y{w) 


h{uj,y{u}))  =  13(0})  ■yiw) 

y(u})  =  \G{en\^ 

<7,2  =  c- 2-2*’ 


(A.IO) 
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Before  proceeding  further,  we  can  now  summarize  the  following  useful  facts  from  [19]: 
factl.  The  sum  of  a  convex  function  with  one  or  more  convex  functions  is  again  convex. 

fact2.  If  f{uj,y{u;))  is  convex  on  [-7r,7r]  x  V,  then,  J[y{u;)]  =  is  convex  on  V,  Hence,  each 

y{(jj)  £  V  that  satisfies  the  necessary  conditions  of  step  2  minimizes  J[y{uj)]  on  V. 

From  the  above  two  facts,  it  is  then  clear  that  to  prove  that  the  solution  (2.14)  is  a  minimizing  curve,  we 
simply  need  to  prove  the  convexity  of  /i{a;,  y{u;))  V  i.  The  convexity  of  fiiuj,  y{u;))  on  [~7r,  tt]  x  V  can  be  established 
by  using  anyone  of  the  following  two  conditions: 

1.  The  following  inequality  must  be  satisfied  V  {uj^y{u))  and  V  {cj,y(o;)  4-^(0?))  £  [— tt, tt]  x  V  : 

fi(u,yiu)  +  v{uj))  -  fi{uj,y{uj))  >  (^^^/t(w,j/(u;)))  •v(a;)  (^-H) 


2.  The  matrix  of  second  partial  derivatives 


fyy  fyy 

ft  f  ’  ' 

VJyy  Jy  y  A 


(>1.12) 


must  be  positive  semidefinite  on  [— tt,  tt]  x  V. 

In  the  above  two  conditions,  all  the  partial  derivatives  are  assumed  to  be  continuous  on  [-7r,7r]  x  P.  The 
notation  y  is  used  fpr  the  derivative  of  y{uj)  with  respect  to  u.  We  use  condition  (A.  11)  to  prove  the  convexity  of 
/2(a;,2/(u;))  and  fsiuJ.yiuj))  and  condition  (A.12)  to  prove  the  convexity  of  /i(u;,y(u;)). 

Convexity  of  f2(tJ,y(u;))  and  £3(0;, y (a?)).  Assume  first  that  /(a;,y(u;))  =  /2(^»y(^))  in  (A.ll).  It  is  then  easy 
to  check,  in  this  case,  that  the  right  hand  side  of  the  equation  is  equal  to  the  left  hand  side.  In  fact,  both  sides 
will  be  equal  to  A  •  5xa'(e*^‘^)  •  t;(ci;).  Similarly,  when  /(u;,  y{uj))  =  /3(a;,y(u;)),  the  right  hand  side  of  (A.ll)  is  equal 
to  the  left  hand  side  of  the  same  equation.  The  two  sides  are,  in  turn,  equal  to  0{uj)  •  v(u;).  This  establish  the 
convexity  of  both  /2(c«j,  y(cj))  and  /3(^)y(^))* 

Convexity  of  fi(u^j  y(i«^))*  When  f{ijA,y{ijj))  =  /i(a;,y(cj)),  then,  we  first  observe  that  the  matrix  in  (A.12)  can 
be  simplified  to  the  following  form: 


fyy  0 
0  0 


(A.13) 


For  this  matrix  to  be  positive 
equivalent  to  proving  that  fyy 
following  equation: 


semidefinite,  the  principal  minors  should  be  non-negative.  From  (A. 13),  this  is 
>  0  V  a;.  Differentiating  /i(u;,y(u;))  twice  with  respect  to  y(cj),  we  obtain  the 


fyy 


(5*i(e>‘^)  •  j/(w)  +  cr|) 


(A.  14) 


Since  all  quantities  in  (A. 14)  are  positive,  then,  the  condition  (A.12)  is  indeed  satisfied  and  /i(u;,  j/(w))  is  convex. 
Using  the  convexity  of  the  above  functions  and  facts  one  and  two,  we  conclude  that  the  solution  (2,14)  is  a  mini¬ 
mizing  extremum.  ■ 
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Appendix  B. 

Using  the  following  set  of  equations: 


e(n)  =  x{n)  —  x{n) 

00 

x{n)  =  z{n)  (g)  h{n)  =  ^  ^^z{n  —  k) 
k=o 

z{n)  =  yin)  +  qin) 
y{n)  =  x(n)  —  ax{n  —  1) 

we  can  eeisily  verify  by  direct  substitution  that  the  error  process  at  the  output  of  the  postfilter  is  given  by: 

00  oo  oo 

e(n)  =  x(n)  -  y'7*x(n-A:)  +  a  •  ^  7*x(n  -  A:  -  1)  -  ^7*g(n  — A:) 

*=o  *=0  *=0 

=  (f  -  l)f;7*x(n-A:)  -  f^f'^qin-k) 

^  Jfc=l  k=0 


The  mean  square  error  expression  is  defined  to  be  £*  =  E{e^{n)],  This,  in  turn  can  be  written  as: 

S  =£[({-  -  l)^7*x(n-A)  -  ^7*9(n- 

X  OO  OO 

■((?  "  “  5ZV9(n-0 

^  ^  1=1  1=0 

This  leist  equation  can  be  simplified  using  the  following  ctssumptions  about  the  noise  process  q{n): 

1.  White  noise  assumption.  E{q{n)  •  q{n  —  k)}  ^  CT^6{n  —  k). 

2.  Variance  constraint  assumption.  The  noise  variance  is  equal  to  where  ay  is  the  variance  of  the 

quantizer  input.  Hence,  =  c2"^^(ilxi(0)(l  +  a^)  —  2aJ2xx(l))- 

3.  Uncorrelai^dness  with  x{n).  The  sequence  x(n)  and  q{n)  are  assumed  to  be  uncorrelated.  Hence, 


-  A:)^ 
)} 


E{x{n)  •  q{n  —  A:)}  =  E{q{n)  •  x{n  —  A:)}  =  0  V  A: 


Based  on  the  above  assumptions,  equation  (B.2)  can  be  therefore  simplified.  The  result  gives  the  follovdng 
expression  for  the  mean  square  error: 


f=c2-2»((l  +  Q2)ii„(0)  -  2aJi,,(l))f^r 
^  '  *=0 

+  (?  - 1)'  i;E7'-"*i?x.(A=-/) 


2k 


k=l  1=1 


(B.3)  . 


This  last  expression  consists  of  two  terms  and  can  be  further  simplified.  The  first  term  of  (B.3)  can  be  rewritten 
as  follows: 


c2-2M(l  +  a^)Rxx{0)  -  2aR^^{l) 


(1-7==) 


(BA) 
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For  the  second  term,  we  make  the  change  of  variable  m  —  k  —  I  to  obtain: 


/  oo  oo  f  \2oo 

(^)  E  (®-5) 

^  ^  ^  /=1  m=0  ^  m=0 

Adding  (B.4)  and  (B.5)  we  obtain  the  mean  square  error  expression  of  theorem  4.1.  ■ 
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Fig.  1.3.  The  equivalent  polyphase  representation  for  a  uniform  M-channel  subband  coder  (SBC). 
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Fig.  2.1.  The  pre  -  and  postfiltering  scheme  with  a  uniform  quantizer  and  an  optimum  postfilter. 


Postfilter 


x(n) 


l—»  x(n) 


Fig.  2.2.  Inserting  a  multiplier  afier  the  prefilter  to  study  the  effect  of  the  quantizer  input  variance. 
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Fig.  4.1.  Coding  gain  curves  for  the  MA(1)  case  :  FIR  prefilter,  OR  postfilter  and  b  —  1. 
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Fig.  4.2.  Coding  gain  curves  for  the  MA(1)  case  :  FIR  prefilter,  HR  postfilter  and  b  =  2. 
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Fig.  4.3.  Coding  gain  curves  for  the  MA(1)  case  :  FIR  prefilter,  DR  postfilter  and  b  =  3 
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Fig.  4.4.  Coding  gain  curves  for  the  MA(1)  case  :  HR  prefilter,  FIR  postfilter  and  b  =  1. 


0.15 


X) 

•D 

.£  0.1 
c 
'co 
O) 

O) 

iO.05 

o 

o 


equal  coefficients 
unequal  coefficients 


0  0.1  0.2  0.3  0.4  0.5  0.6  0.7  0.8  0.9 

parameter  theta 


Fig.  4.5.  Coding  gain  curves  for  the  MA(1)  case  :  EER  prefilter,  FIR  postfilter  and  b  —  2. 
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Fig.  4.6.  Coding  gain  curves  for  the  MA(1)  case  :  IIR  prefilter,  FIR  postfilter  and  b  —  3 
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Fig.  4.8.  Coding  gain  curves  for  the  AR(1)  case  :  FIR  prefilter,  HR  postfilter  and  b  =  2. 
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Fig.  4.9.  Coding  gain  curves  for  the  AR(1)  case  :  FIR  prefilter,  HR  postfilter  and  b  =  3. 
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Fig.  4.10.  Coding  gain  curves  for  the  AR(1)  case  ;  HR  prefilter,  FIR  postfilter  and  b  =  1. 
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Fig.  4.11.  Coding  gain  curves  for  the  AR(1)  case  :  HR  prefilter,  FIR  postfilter  and  b  =  2. 
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Fig.  4.12.  Coding  gain  curves  for  the  AR(1)  case  :  HR  prefilter,  FIR  postfilter  and  b  =  3. 
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Fig.  5.1.  An  M-channel  non-uniform  subband  coder  (SBC). 
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Fig.  5.2.  The  optimum  uniform  orthonorraal  FB  with  the  general  pre  -  and  post  filtering  scheme 
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Fig.  5.3.  The  power  spectral  density  for  the  input  of  example  5.1. 
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Table  1.  The  coding  gain  obtained  from  first  order  filters  for  the  AR(5)  case  of  example  4.3.3 
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